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Ada  Performance  Benchmarks  on  the 
MicroVAX  II:  Summary  and  Results 
Version  1 .0 


Abstract:  This  report  documents  the  results  obtained  from  running  the  University 
of  Michigan  and  the  ACM  SIGAda  Performance  Issues  Working  Group  (PIWG) 
Ada  performance  benchmarks  on  a  DEC  VAXELN  MicroVAX  II  using  the  DEC 
VAXELN  Ada  compiler.  A  brief  description  of  the  benchmarks  and  the  test  envi¬ 
ronment  is  followed  by  a  discussion  of  some  problems  encountered  and  lessons 
learned.  The  output  of  each  benchmark  program  is  also  included. 


1.  Summary 


c 


e.. 


The  primary  purpose  of  the  Ada  Embedded  Systems  Testbed  (AEST)  Project  at  the  Soft¬ 
ware  Engineering  Institute  (SEI)  is  to  develop  a  solid  in-house  support  base  of  hardware, 
software,  and  personnel  to  permit  the  investigation  of  a  wide  variety  of  issues  related  to 
software  development  for  real-time  embedded  systems. )  Two  of  the  most  crucial  issues  to 
be  investigated  are  the  extent  and  quality  of  the  facilities  provided  by  Ada  runtime  support 
environments.  The  SEI  support  basewilUn§Jje-as§essm  e  n  ts  possible  of  the  readiness  of 
the  Ada  language  and-Ada  tootsTtrdevelop^mbedded  systems. 

The  benchmarking/instrumentation  subgroup  was  formed  to: 


I }  Collect  and  run  available  Ada  benchmark  programs  from  a  variety  of  sources 
on  a  variety  of  targets^ 

j  Identify  gaps  in  the  coverage  and  fill  them  with  new  test  programs. 

3  }  Review  the  measurement  techniques  used  and  provide  new  ones'* if  necessary.  r>  ' 
u  y  Verify  software  timings  by  inspection  and  with  specialized  test  instruments,  jj  , 


l 


This  report  documents  the  results  obtained  from  running  Ada  performance  benchmarks  on  a 
DEC  VAXELN  MicroVAX  II  using  the  DEC  VAXELN  Ada  compiler.  The  benchmarks  were 
the  University  of  Michigan  Ada  benchmarks  and  the  ACM  SIGAda  Performance  Issues 
Working  Group  (PIWG)  Ada  benchmarks  (excluding  the  compilation  tests).  A  description  of 
these  suites  and  the  reasons  for  choosing  them  are  given  in  [9].  The  benchmarks  focus 
largely  on  the  execution  time  of  specific  features  of  the  Ada  language;  they  do  not,  for  ex¬ 
ample,  measure  the  efficiency  or  the  size  of  the  generated  object  code.  A  brief  description 
of  the  benchmarks  and  the  test  environment  is  followed  by  a  discussion  of  some  problems 
encountered  and  lessons  learned.  The  results  obtained  from  running  the  entire  Michigan 
and  PIWG  benchmark  suites  are  contained  in  the  appendices  to  this  report.  Note  that  the 
caveats  discussed  in  the  body  of  the  report  must  be  borne  in  mind  when  examining  these 
results. 
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2.  Discussion 


2.1.  The  University  of  Michigan  Ada  Benchmarks 

The  University  of  Michigan  benchmarks  concentrate  on  techniques  for  measuring  the  perfor¬ 
mance  of  individual  features  of  the  Ada  programming  language.  The  development  of  the 
real-time  performance  measurement  techniques  and  the  interpretation  of  the  benchmark 
results  are  based  on  the  Ada  notion  of  time.  An  article  by  the  Michigan  team  [4]  begins  by 
reviewing  the  Ada  concept  of  time  and  the  measurement  techniques  used  in  the 
benchmarks.  The  specific  features  measured  are  then  discussed,  followed  by  a  summary  of 
the  results  obtained  and  an  appraisal  of  these  results.  A  follow-up  letter  about  the  Michigan 
benchmarks  appears  in  [3]. 

2.2.  The  Performance  Issues  Working  Group  (PIWG)  Ada  Benchmarks 

The  PIWG  benchmarks  comprise  many  different  Ada  performance  tests  that  were  either  col¬ 
lected  or  developed  by  PIWG  under  the  auspices  of  the  ACM  Special  Interest  Group  on  Ada 
(SIGAda).  In  addition  to  language  feature  tests  similar  to  the  Michigan  benchmarks,  the 
PIWG  suite  contains  composite  synthetic  benchmarks  such  as  Whetstone  [5],  [10]; 
Dhrystone  [11];  and  a  number  of  tests  to  measure  speed  of  compilation.  PIWG  distributes 
tapes  of  the  benchmarks  to  interested  parties  and  collects  and  publishes  the  results  in  a 
newsletter.  Workshops  and  meetings  are  held  during  the  year  to  discuss  new  benchmarks 
and  suggestions  for  improvements  to  existing  benchmarks.1 

2.3.  Testbed  Hardware  and  Software 

The  hardware  used  for  benchmarking  was  a  DEC  Micro  VAX  II  host,  running  MicroVMS  4.4, 
linked  to  a  MicroVAX  II  target.  The  target  had  five  megabytes  of  RAM.  a  dual  floppy  disk 
drive,  and  was  linked  to  the  host  via  DECnet.  Programs  on  the  target  machine  ran  under 
control  of  the  VAXELN  kernel,  an  executive  providing  job  and  process  scheduling  on  a 
prioritized  pre-emptive  basis  [6],  [7].  The  hardware  and  software  can  be  summarized  as  fol¬ 
lows: 

Host:  DEC  MicroVAX  II,  njnning  MicroVMS  4.4 

Compiler:  DEC  VAXELN  Ada,  release  1 .1  (DEC  VAX  Ada  1 .3),  ACVC  1 .7 

Target:  DEC  MicroVAX  II  with  VAXELN  2.3, 5Mb  RAM 

The  complete  VAXELN  tool  kit  is  a  software  product  for  the  development  of  real-time  sys¬ 
tems  for  VAX  processors.  It  provides  most  of  the  standard  VAX/VMS  development  tools, 
such  as  the  VAX  Ada  Compilation  System  (ACS),  and  includes  a  VAXELN  Ada  runtime 
library  and  a  VAXELN  remote  debugger.  The  remote  debugger  can  be  used  to  download 
and  activate  programs  on  the  target,  whether  or  not  they  have  been  compiled  with  the  de- 


’The  benchmarks  came  from  the  PIWG  distribution  tape  known  as  TAPE_8_31_86.  The  name,  address,  and 
telephone  number  of  the  current  chairperson  of  the  PIWG  can  be  found  in  Ada  Letters,  a  bimonthly  publication  of 
SIGAda,  the  ACM  Special  Interest  Group  on  Ada 
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bugger  option.  The  VAXELN  Ada  compiler  [8]  is  substantially  identical  to  the  VAX  Ada  com¬ 
piler,  with  the  exception  of  some  pragmas  (e.g.,  VAXELN  Ada  does  not  support  the 
TIME_SUCE  pragma)  and  VAXELN  Ada’s  lack  of  relative  and  indexed  file  support.  The 
host-based  development  tools  are  used  to  create  an  application  program  and  build  a 
VAXELN  executable  target  system  that  can  be  booted  on  the  target  machine  from  a  floppy 
disk  or  tape,  or  downloaded  to  the  target  via  DECnet. 

2.4.  Running  the  Benchmarks 

Both  the  Michigan  and  PIWG  benchmark  suites  contained  command  files  for  compiling  and 
running  the  tests  under  VAX/VMS.  The  Michigan  benchmarks  had  a  command  file  for  each 
category  of  tests  (e.g.,  one  for  rendezvous  tests,  one  for  exception  handling  tests),  whereas 
the  PIWG  suite  had  a  single  command  file  that  couid  be  adapted  to  run  any  test.  The 
Michigan  command  files  were  run  through  a  ’’pre-processor"  command  file  that  produced  an 
expanded  command  file  capable  of  building  and  downloading  a  VAXELN  executable  sys¬ 
tem.  The  benchmark  output,  which  normally  would  have  appeared  on  the  target  machine's 
console,  was  re-routed  to  a  file  on  the  host.  It  was  also  possible  to  create  a  bootable  floppy 
disk;  as  a  test,  several  executable  VAXELN  images  were  created  as  both  a  bootable  floppy 
disk  and  a  file  to  be  downloaded  from  the  host.  Virtually  no  variation  in  the  results  produced 
by  either  method  was  observed,  so  the  downloadable  file  became  the  preferred  method 
since  it  could  be  fully  controlled  from  the  host. 

All  benchmarks  were  compiled  with  VAXELN  Ada’s  default  optimizations  turned  on.2  The 
benchmarks  contained  code  to  prevent  the  language  feature  of  interest  from  being  op¬ 
timized  away.  Runtime  checks  were  not  suppressed,  and,  apart  from  the  Michigan 
exception-handling  problem  noted  below,  the  benchmarks’  source  code  was  not  modified  in 
any  way.  Benchmark  results  are  listed  in  the  appendices. 

2.5.  Problems  Encountered  and  Lessons  Learned 

A  number  of  minor  problems  were  encountered  during  the  running  of  the  benchmarks;  these 
are  noted  below  in  the  appropriate  results  section.  The  one  major  problem  that  arose  only 
appeared  after  most  of  the  Michigan  tests  had  been  run:  negative  time  values  were  pro¬ 
duced  for  some  of  the  tests  (Dynamic  Storage  Allocation  and  Subprogram  Overhead  tests). 
An  investigation  revealed  that  the  VAXELN  paging  mechanism  lengthened  the  execution 
times  of  loops  that  spanned  a  page  boundary.  (Physical  memory  on  the  VAXELN  target  is 
divided  into  512-byte  pages;  however,  no  swapping  to  disk  took  place  since  disk  support 
was  not  included.  The  benchmarks  were  entirely  resident  in  memory.)  Thus  the  control 
loop  of  some  benchmarks  would  actually  take  longer  to  run  than  the  test  loop,  and  the  ex¬ 
ecution  time  of  the  language  feature  being  measured  (expressed  as  the  difference  of  the 


2The  compiler  performs  a  number  of  standard  optimizations,  including:  elimination  of  common  sub¬ 
expressions;  removal  of  invariant  computations  from  bops;  in-line  code  expansbn;  gbbal  assignment  of  vari¬ 
ables  to  registers;  peephole  optimization  of  instruction  sequences;  and  elimination  of  dead  code.  If  these 
optimizations  are  not  desired,  the  user  must  explicitly  disable  them  by  invoking  an  option  with  the  compile 
command. 
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test  and  control  times)  would  sometimes  be  negative.  A  more  detailed  discussion  of  the 
so-called  "dual  loop  problem''  can  be  found  in  [1].  A  complete  report  on  the  problems  en¬ 
countered  during  the  AEST  benchmarking  effort,  and  a  discussion  of  other  possible  bench¬ 
marking  pitfalls,  Is  contained  in  [2]. 

Another  interesting  issue  is  the  accuracy  of  times  reported  by  the  PIWG  benchmarks.  One 
of  the  PIWG  benchmark  support  packages,  A000032.ADA,  contains  the  body  of  the  ITERA¬ 
TION  package.  This  package  is  called  by  a  benchmark  program  to  calculate,  among  other 
things,  the  minimum  duration  for  the  test  loop  of  a  benchmark  run.  The  minimum  duration  is 
computed  to  be  the  larger  of  1  second,  100  times  System.Tick,  and  100  times 
StandardDuration’Small.  The  idea  appears  to  be  (a)  to  run  the  benchmark  for  enough 
iterations  to  overcome  the  problem  of  the  relatively  coarse  resolution  of  the  Calendar.Clock 
function,  and  (b)  to  provide  a  relative  accuracy  of  one  percent  or  better.  The  times  reported 
by  the  benchmark  programs  are  printed  with  an  accuracy  of  one  tenth  of  a  microsecond; 
however,  merely  running  the  test  for  a  specific  minimum  duration  does  not  guarantee  this 
degree  of  accuracy.  If  the  clock  resolution  is  10  milliseconds,  for  example,  and  the  desired 
accuracy  is  to  within  1  microsecond,  then  the  test  should  be  run  for  1 0,000  iterations.  For 
Ada  language  features  that  execute  in  tens  of  microseconds,  running  for  a  specific  duration 
may  ensure  enough  iterations  for  accuracy  to  within  one  microsecond;  this  is  not  so  for  lan¬ 
guage  features  that  take  longer. 

In  general,  the  accuracy  of  the  PIWG  and  Michigan  benchmarks  is  to  within  one  tick  of 
Calendar.Clock  divided  by  the  number  of  iterations  of  the  benchmark  (see  the  Basic  Meas¬ 
urement  Accuracy  section  of  the  University  of  Michigan  report).  The  University  of  Michigan 
benchmarks  typically  run  for  10,000  iterations,  and  so  are  accurate  to  within  1  microsecond 
for  VAXELN  Ada  (10  millisecond  Calendar.Clock  resolution).  The  task  creation  tests  and 
some  of  the  dynamic  storage  allocation  tests  run  for  fewer  iterations,  probably  because  of 
the  amount  of  storage  they  use  up;  the  reduced  accuracy  is  noted  in  the  appropriate  sec¬ 
tions.  Also,  the  source  of  the  exception-handling  tests  had  to  be  modified  to  reduce  the 
number  of  iterations  so  that  the  test  would  actually  run.  For  the  PIWG  tests,  a  table  of 
iteration  counts  and  resultant  accuracy  is  provided  in  the  PIWG  results  appendix. 

Comparison  of  the  results  from  the  most  closely  equivalent  PIWG  and  Michigan  benchmarks 
has  been  hindered  by  the  accuracy  problem  and  the  dual  loop  problem.  Even  when  the 
correction  factors  are  applied  to  take  care  of  the  former,  the  precise  effects  of  the  dual  loop 
problem  on  each  benchmark  program  are  not  known.  It  is  clear  that  more  work  needs  to  be 
done  to  resolve  such  problems. 

The  VAXELN  benchmarking  effort  was  essentially  a  learning  experience.  The  major  les¬ 
sons  learned  were: 

•  It  is  very  important  to  check  the  underlying  assumptions  incorporated  in  the 
benchmark  design  before  attempting  to  use  the  benchmark.  A  simple  example 
of  such  a  check  is  a  ’’calibration*  routine  to  check  whether  or  not  a  dual  loop 
test  with  textually  identical  loops  will  zero  out. 
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•  Even  when  few  or  no  problems  are  encountered  during  the  running  of  the 
benchmarks,  the  results  should  be  checked  for  reasonableness,  especially  if 
the  times  reported  are  different  from  heuristically  calculated  figures. 

•  Inspection  of  generated  assembly  code  (however  distasteful  this  might  be  to  an 
Ada  aficionado)  can  turn  up  dues  to  puzzling  results.  Once  problems  start  oc¬ 
curring,  knowledge  of  the  machine's  instruction  set  architecture  and  underlying 
hardware  can  prove  very  useful. 

The  major  result  of  the  VAXELN  MicroVAX  benchmarking  effort,  therefore,  is  not  a  list  of 
numbers  to  be  taken  at  face  value;  rather,  it  is  an  appreciation  of  the  problems  and  pitfalls 
facing  the  would-be  benchmarker.  Analysis  of  the  results  from  the  VAXELN  and  other 
cross-compilers  and  target  systems,  as  well  as  analysis  of  the  benchmarks  themselves,  will 
be  one  of  the  main  items  of  business  in  the  AEST  Project’s  second  year. 
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Appendix  A:  Results:  University  of  Michigan 
Benchmarks 


In  the  results  presented  below,  certain  lines  of  output  have  been  omitted  for  the  sake  of 
brevity.  Many  of  the  Michigan  tests  print  out  lines  of  "raw  data,”  and  the  command  files 
sometimes  run  a  particular  test  many  times;  these  are  the  lines  that  have  been  omitted. 
Also,  some  of  the  headings  have  been  split  over  two  lines  to  make  them  fit  this  document. 


A.a.  Clock  Calibration  and  Overhead 

One  of  the  Michigan  "tests"  merely  prints  the  values  of  System.Tick  and 
Standard.Duratlon'Small.  For  VAXELN  Ada  these  are: 

System  Ticks  0.009948730468750  seconds 

Duration  Smalls  0.000061035156250  seconds 


Thus  System.Tick  is  approximately  10  milliseconds,  and  Duratlon’Small  is  approximately 
61  microseconds.  The  clock  calibration  test  determines  the  resolution  of  the 
Calendar.Clock  function.  As  can  be  seen  from  the  data  below,  the  resolution  is  10  mil¬ 
liseconds.  the  value  of  System.Tick. 


Output  o£  second  differencing 
Humber  zeros  previous: 

Time  difference  (in  seconds) : 
Number  zeros  previous: 

Time  difference  (in  seconds) : 
Number  zeros  previous: 

Time  difference  (in  seconds) : 


is  as  follows: 

94 

0.009948730468750 

0 

-0.009948730468750 

112 

0.009948730468750 


Number  zeros  previous: 

Time  difference  (in  seconds) : 
Number  zeros  previous: 

Time  difference  (in  seconds) : 
Number  of  iterations  »  10000 


112 

0.009948730468750 

0 

-0.009948730468750 


It  should  be  noted  that  the  negative  times  above  are  a  legitimate  result  of  the  test  and  have 
nothing  to  do  with  the  dual  loop  problem  discussed  earlier. 

The  test  to  measure  the  overhead  associated  with  calling  Calendar.Clock  produced  consis¬ 
tently  repeatable  results,  so  only  one  line  of  output  is  shown: 

Clock  function  calling  ovarhaad  :  84.00  microseconds 
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A.b.  Task  Rendezvous 


For  this  test,  a  procedure  calls  the  single  entry  point  of  a  task;  no  parameters  are  passed, 
and  the  called  task  executes  a  simple  accept  statement.  According  to  the  Michigan  report, 
it  is  assumed  that  such  a  rendezvous  will  involve  at  least  two  context  switches. 

Rendezvous  tine  :  No  psransters  passed 
Number  of  iterations  *  10000  | 


Task  rendezvous  tins  :  1585.0  microseconds 


A.c.  Task  Creation 

These  tests  measure  the  composite  time  taken  to  elaborate  a  task's  specification,  activate 
the  task,  and  terminate  the  task.  The  coarse  resolution  of  the  clocks  available  at  the  time 
the  tests  were  developed  did  not  allow  for  measurement  of  the  Individual  components  of  the 
test.  Also,  because  these  tests  are  run  for  100  iterations,  the  reported  times  are  accurate  to 
100  microseconds,  or  0.1  milliseconds. 

To  obtain  the  third  test  result  below,  the  VAXELN  pool  size  (which  determines  the  number  of 
\/AXELN  objects  that  can  be  in  simultaneous  use)  had  to  be  increased  from  the  default  of 
384  blocks  to  1024  blocks  (a  block  is  512  bytes). 

Task  elaborate,  activate,  and  terminate  time: 

Declared  object,  no  type 
Number  of  iterations  =  100 

Task  elaborate,  activate,  terminate  time:  9.7  milliseconds 


Task  elaborate,  activate,  and  terminate  time: 

Declared  object,  task  type 
Number  of  Iterations  =  100 

Task  elaborate,  activate,  terminate  time:  9.5  milliseconds 


Task  elaborate,  activate,  and  terminate  time: 

NEW  object,  task  type 
Number  of  iterations  =  100 

Task  elaborate,  activate,  terminate  time:  8.9  milliseconds 
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A.d.  Exception  Handling 

The  exception-handling  benchmark  kept  crashing  with  a  STORAGE_ERROR  exception  de¬ 
spite  many  attempts  to  tailor  the  storage  parameters  of  the  VAXELN  system  build  process. 
Eventually  it  was  made  to  run  by  reducing  the  number  of  iterations  of  the  test  from  1000  to 
100.  This  was  the  only  case  where  benchmark  code  had  to  be  modified.  A  possible  reason 
for  the  problem  (see  the  Memory  Management  section)  is  the  lack  of  storage  reclamation 
(garbage  collection)  procedures;  space  used  during  exception-handling  probably  remains  al¬ 
located  after  the  exception-raising  procedure  exits.  The  reduced  number  of  iterations 
means  that  the  times  shown  below  are  accurate  only  to  within  100  microseconds. 

Number  of  iterations  =  100 


Exception  Handler  Testa 


Exception  raised  and  handled  in  a  block 


0.0  uSEC. 
799.6  uSEC. 
999.8  uSEC. 

999.8  uSEC. 

499.9  uSEC. 
999.8  uSEC. 
999.8  uSEC. 


User  defined,  not  raised 
User  defined 

Constraint  error,  implicitly  raised 
Constraint  error,  explicitly  raised 
Numeric  error,  implicitly  raised 
Numeric  error,  explicitly  raised 
Tasking  error,  explicitly  raised 


Exception  raised  in  a  procedure  and  handled  in  the 
calling  unit 


0.0  uSEC. 
900.3  uSEC. 
1000.4  uSEC. 
1000.4  uSEC. 

800.2  uSEC. 
1000.4  uSEC. 
1000.4  uSEC. 


User  defined,  not  raised 
User  defined 

Constraint  error,  implicitly  raised 
Constraint  error,  explicitly  raised 
Numeric  error,  implicitly  raised 
Numeric  error,  explicitly  raised 
Tasking  error,  explicitly  raised 
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A.e.  Time  and  Duration  Math 

In  the  results  below,  the  lines  flagged  with  an  asterisk  are  from  tests  that  had  to  be  run 
individually  to  get  them  to  work.  When  included  in  a  command  file  that  ran  all  of  the  tests 
sequentially,  these  two  tests  would  always  cause  VAXELN  Ada  to  generate  a  runtime  error 
message  saying  that  the  "computed  year  is  not  in  the  range  of  subtype  YEAR_NUMBER." 

Number  of  Iterations  «  10000 


Time  and  Duration  Math 


uSKC .  Operat ion 


90.00 

94.00 

89.00 

94.00 

*  93.00 

*  94.00 
103.00 

3.00 

3.00 

3.00 

4.00 

3.00 

4.00 

3.00 

3.00 


Time  :*  Var_time  +  var_dur at ion 

Time  :  *  Varjtime  4-  const_duration 

Time  :«  Var~du  ration  4-  var_time 

Time  :■  Const_duration  4  var__time 

Time  : *  Var__tima  -  varjduration 

Time  :■  Var__time  -  const_duration 

Duration  :«  Var_time  -  var_time 

Duration  : *  Var_duration  +  var_duration 

Duration  :«  Var_duration  4-  const_duration 

Duration  :«  Const_duration  4-  var_duration 

Duration  :«  Const_duration  4-  const_duration 

Duration  :«  Var_duration  -  var_duration 

Duration  : »  Var_duration  -  const_duration 

Duration  :*  Cons t_durat ion  -  var_duration 

Duration  : =  Const  duration  -  const  duration 
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A.f.  Delay  Statement  Tests 

For  VAXELN  Ada,  System.Tlck  is  10  milliseconds  and  Standard.Duratlon'Small  is  61 
microseconds.  In  the  results  below,  the  desired  delay  times  start  at  Duratlon'Small  and 
increment  by  Duratlon’Small.  The  actual  delay  time  of  0.01996  seconds  is  twice 
Syatam.Tlck;  0.02997  is  three  times  System.Tlck;  and  0.03998  is  four  times  System.Tlck. 
Thus  the  smallest  delay  that  can  be  achieved  by  a  delay  statement  in  the  VAXELN  imple¬ 
mentation  is  approximately  20  milliseconds. 

Number  of  iterations  »  1 


For  case  number 
Desired  delay  time: 
Actual  delay  time: 

1 

0.00006 

0.01996 

seconds 

seconds 

For  case  number 
Desired  delay  time: 
Actual  delay  time: 

• 

• 

2 

0.00012 

0.01996 

seconds 

seconds 

• 

For  case  number 
Desired  delay  time: 
Actual  delay  time: 

164 

0.01001 

0.01996 

seconds 

seconds 

For  case  number 
Desired  delay  time: 
Actual  delay  time: 

• 

• 

165 

0.01007 

0.02997 

seconds 

seconds 

For  case  number 
Desired  delay  time: 
Actual  delay  time: 

328 

0.02002 

0.02997 

seconds 

seconds 

For  case  number 
Desired  delay  time: 
Actual  delay  time: 

329 

0.02008 

0.03998 

seconds 

seconds 

For  case  number 
Desired  delay  time: 
Actual  delay  time: 


350 

0.02136  seconds 
0.03998  seconds 


A.g.  Dynamic  Storage  Allocation 

There  are  three  categories  of  allocation  measured  by  these  tests: 

1.  Fixed  Storage  Allocation:  The  objects  are  declared  locally  in  a  subprogram  or 
declare  block;  the  storage  required  is  known  at  compile  time  but  is  allocated 
at  run  time. 

2.  Variable  Storage  Allocation:  Same  as  for  fixed  allocation,  but  the  storage  re¬ 
quired  (e.g.,  in  the  case  of  an  array  with  variable  bounds)  is  not  known  at  com¬ 
pile  time. 

3.  Explicit  Dynamic  Allocation:  Storage  is  allocated  via  the  new  allocator. 

These  tests  were  the  first  to  exhibit  symptoms  of  the  "dual  loop”  problem  (negative  times) 

referred  to  earlier  in  this  report. 

Number  of  iterations  =  10000 


Dynamic  Allocation  in  a  Declarative  Region 


I  Integer 
I  Integer 
I  String 
|  String 
|  String 
|  Enumeration 
(Enumeration 
|  Enumeration 
(Integer  array 
(Integer  array 
(Integer  array 
|  Integer  array 
1 1-D  Dynamically  bounded 
1 1-D  Dynamically  bounded 
(2-D  Dynamically  bounded 
|2-D  Dynamically  bounded 
1 3-D  Dynamically  bounded 
1 3-D  Dynamically  bounded 
(Record  of  integer 
(Record  of  integer 
I  Record  of  integer 


array 

array 

array 

array 

array 

array 


1 

10 

100 

1000 

1 

10 

1 

100 

1 

1000 

1 

10 

100 
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Because  these  tests  only  iterate  1000  times,  tee  reported  times  are  accurate  to  within  10 
microseconds,  rather  than  1  microsecond. 

Wusfeer  of  iterations  ■  1000 


Dynastic  Allocation  with  HEW  allocator 


1 

1 

#  Dsdartd 

1  *ypa 

|  Declared 

1 

1 

Size  of 
Object 

T 

1 

| Integer 

1 

1 

i 

1 

(Enumeration 

1 

1 

i 

1 

| Record  of  integer 

1 

1 

i 

1 

(Record  of  integer 

1 

10 

i 

1 

| Record  of  integer 

1 

100 

i 

1 

(Record  of  integer 

1 

20 

i 

1 

| Record  of  integer 

1 

5 

i 

1 

(Record  of  integer 

1 

50 

i 

1 

| Integer  array 

1 

1 

i 

1 

(Integer  array 

1 

10 

i 

1 

I Integer  array 

1 

100 

i 

1 

(Integer  array 

1 

1000 

i 

1 

| String 

1 

1 

» 

1 

(String 

1 

10 

i 

1 

j  String 

1 

100 

i 

1 

(1-D  Dynamically  bounded 

array  | 

1 

i 

1 

| 1-D  Dynamically  bounded 

array  | 

10 

i 

1 

| 2-D  Dynamically  bounded 

array  | 

1 

i 

1 

| 2-D  Dynamically  bounded 

array  | 

100 

i 

1 

| 3-D  Dynamically  bounded 

array  | 

1 

i 

1 

(3-D  Dynamically  bounded 

array  | 

1000 

Tima 

(aicrosoc . ) 


280.0 

280.0 

280.0 

290.0 

280.0 

280.0 

290.0 

290.0 

290.0 

290.0 

290.0 
290.0 
290.0 
290.0 
300.0 
310.0 
310.0 
340.0 
140.0 
390.0 
390.0 
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A.h.  Subprogram  Overhead 

Several  kinds  of  subprogram  overhead  benchmarks  are  provided.  They  measure  the  over¬ 
head  involved  in  entering  and  exiting  a  subprogram  with  no  parameters,  with  various  num¬ 
bers  of  scalar  parameters,  and  with  various  numbers  of  composite  objects  (arrays  and 
records)  as  parameters.  Tests  are  also  provided  to  measure  the  overhead  associated  with 
passing  constraint  information  to  subprograms  whose  formal  parameters  are  of  an  uncon¬ 
strained  composite  type.  All  of  the  tests  include  passing  parameters  in  all  three  modes:  In, 
out,  and  In  out. 

All  of  the  tests  also  measure  the  difference  in  overhead  between  calling  subprograms  in 
different  packages  and  calling  subprograms  in  the  same  package.  For  intra-package  calls, 
there  are  also  versions  of  the  tests  to  measure  the  overhead  of  using  the  INLINE  pragma,  if 
the  pragma  is  supported.3  Finally,  all  the  tests  for  inter-  and  intra-package  calls  are 
repeated  with  the  subprograms  appearing  as  part  of  a  generic.  These  tests  determine  the 
overhead  associated  with  executing  generic  instantiations  of  the  code. 

The  subprogram  overhead  tests  were  the  second  major  source  of  negative  time  values. 
The  negative  numbers  for  these  tests  were  generally  a  lot  smaller  than  those  produced  by 
the  dynamic  storage  allocation  tests. 

Subprogram  Overhead  (non-generic) 

Number  of  iterations  *  10000  *  10 


Time 

Direction 

I#  Passed 

1  Type 

|  Sire 

of  | 

(microsec . ) 

Passed 

lin 

Call 

|  Passed 

| Passed 

Var  | 

|  0.8 

1 

0 

1 

1 

1 

|  0.2 

I 

1 

1 

|  INTEGER 

1 

1 

|  0.0 

O 

1 

1 

| INTEGER 

1 

1 

|  0.7 

I  0 

1 

1 

| INTEGER 

1 

1 

|  -0.1 

I 

1 

10 

| INTEGER 

1 

1 

1  0.1 

0 

1 

10 

| INTEGER 

1 

1 

|  13.2 

I  0 

1 

10 

| INTEGER 

1 

1 

|  134.6 

I 

1 

100 

| INTEGER 

1 

1 

|  197.4 

0 

1 

100 

| INTEGER 

1 

1 

I  303.6 

I  0 

1 

100 

| INTEGER 

1 

1 

continued  . . . 


3VAXELN  Ada  supports  the  INLINE  pragma. 
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-0.2 

0.0 

0.6 

0.4 

-1.4 

2.0 

135.3 

188.8 

294.5 

1.7 
-1.8 
-0.1 

0.1 

0.0 

0.8 

-1.2 

0.0 

0.4 

0.2 

0.1 

0.2 

-0.4 

0.5 

2.8 
-0.2 
-0.2 

1.5 

-0.3 

-0.3 

0.1 

-1.2 

0.2 

0.1 

0.1 

-0.4 

0.1 


1 

(ENUMERATION 

1 

1 

| ENUMERATION 

1 

1 

(ENUMERATION 

1 

10 

(ENUMERATION 

1 

10 

(ENUMERATION 

1 

10 

(ENUMERATION 

1 

100 

(ENUMERATION 

1 

100 

(ENUMERATION 

1 

100 

(ENUMERATION 

1 

1 

(ARRAY  Of  INTEGER  | 

1 

1 

| ARRAY  of  INTEGER  | 

1 

1 

| ARRAY  of  INTEGER  | 

1 

1 

| ARRAY  of  INTEGER  | 

10 

1 

| ARRAY  of  INTEGER  | 

10 

1 

|  ARRAY  of  INTEGER  ( 

10 

1 

(ARRAY  of  INTEGER  | 

100 

1 

| ARRAY  of  INTEGER  | 

100 

1 

| ARRAY  of  INTEGER  | 

100 

1 

| RECORD  of  INTEGER  | 

1 

1 

| RECORD  of  INTEGER  | 

1 

1 

(RECORD  of  INTEGER  | 

1 

1 

| RECORD  of  INTEGER  | 

100 

1 

| RECORD  of  INTEGER  | 

100 

1 

| RECORD  of  INTEGER  | 

100 

1 

(UNCONSTRAINED 

ARRAY  | 

1 

1 

(UNCONSTRAINED 

ARRAY  | 

1 

1 

(UNCONSTRAINED 

ARRAY  | 

1 

1 

| UNCONSTRAINED 

ARRAY  | 

100 

1 

(UNCONSTRAINED 

ARRAY  | 

100 

1 

(UNCONSTRAINED 

ARRAY  | 

100 

1 

| UNCONSTRAINED 

RECORD | 

1 

1 

(UNCONSTRAINED 

RECORD | 

1 

1 

(UNCONSTRAINED 

RECORD | 

1 

1 

(UNCONSTRAINED 

RECORD] 

100 

1 

(UNCONSTRAINED 

RECORD | 

100 

1 

(UNCONSTRAINED 

RECORD! 

100 
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Subprogram  Overhead  (Inline) 


Number  of  itarations  *  10000  *  10 


Tima 

(microaac . ) 

Direction | 

Parsed  | 

#  Passed 

in  Call 

|  Type  |  Sire  of  | 

|  Passed  | Passed  Var| 

1 

0.9 

1 

0 

1  1 

1 

1 

0.3 

I  | 

1 

| INTEGER  | 

1 

1 

-0.1 

0  | 

1 

| INTEGER  | 

1 

1 

0.7 

I  0  | 

1 

| INTEGER  | 

1 

1 

0.1 

I  | 

10 

| INTEGER  | 

i 

1 

-0.2 

0  | 

10 

| INTEGER  | 

1 

1 

13.2 

I_0  | 

10 

| INTEGER  | 

1 

1 

134.5 

I  | 

100 

| INTEGER  | 

1 

1 

197.5 

0  | 

100 

1  INTEGER  | 

1 

I 

303.9 

I  0  1 

100 

| INTEGER  | 

! 

1 

-0.2 

I  1 

1 

(ENUMERATION  | 

1 

1 

-0.1 

0  | 

1 

(ENUMERATION  | 

1 

1 

0.7 

I  0  1 

1 

(ENUMERATION  | 

1 

1 

0.2 

I  1 

10 

(ENUMERATION  ( 

1 

1 

-1.5 

0  | 

10 

(ENUMERATION  | 

1 

1 

2.1 

I  0  1 

10 

(ENUMERATION  | 

1 

1 

135.2 

I  1 

100 

(ENUMERATION  | 

1 

1 

188.6 

0  | 

100 

(ENUMERATION  | 

1 

1 

294.2 

I  0  1 

100 

(ENUMERATION  | 

1 

1 

1.7 

X  1 

1 

| ARRAY  of  INTEGER  | 

1 

1 

1 

-1.9 

o  t 

1 

|  ARRAY  of  INTEGER  | 

1 

1 

1 

-0.1 

I_0  | 

1 

| ARRAY  of  INTEGER  | 

1 

1 

1 

0.0 

I  1 

1 

(ARRAY  Of  INTEGER  | 

10 

1 

1 

-0.4 

o  | 

1 

| ARRAY  Of  INTEGER  | 

10 

1 

1 

0.9 

I  0  1 

1 

(ARRAY  Of  INTEGER  | 

10 

1 

1 

-1.4 

I  1 

1 

| ARRAY  of  INTEGER  | 

100 

1 

1 

-0.2 

o  | 

1 

| ARRAY  Of  INTEGER  | 

100 

1 

1 

0.4 

I  0  1 

1 

| ARRAY  Of  INTEGER  | 

100 

1 

1 

0.0 

I  1 

1 

(RECORD  of  INTEGER  | 

1 

1 

1 

0.1 

o  t 

1 

| RECORD  of  INTEGER  | 

1 

I 

1 

0.1 

I  0  1 

1 

| RECORD  of  INTEGER  | 

1 

1 

1 

-0.6 

I  1 

1 

(RECORD  of  INTEGER  | 

100 

1 

1 

0.6 

o  | 

1 

| RECORD  of  INTEGER  | 

100 

1 

1 

2.9 

I_0  1 

1 

| RECORD  of  INTEGER  | 

100 

1 

.  . .continued 
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1  | UNCONSTRAINED 
1  | UNCONSTRAINED 
1  | UNCONSTRAINED 
1  | UNCONSTRAINED 
1  |  UNCONSTRAINED 
1  (UNCONSTRAINED 
1  (UNCONSTRAINED 
1  (UNCONSTRAINED 
1  (UNCONSTRAINED 
1  (UNCONSTRAINED 
1  | UNCONSTRAINED 
1  (UNCONSTRAINED 


ARRAY  | 

1 

ARRAY  ( 

1 

ARRAY  | 

1 

ARRAY  | 

100 

ARRAY  | 

100 

ARRAY  ( 

100 

RECORD | 

1 

RECORD | 

1 

RECORD  | 

1 

RECORD | 

100 

RECORD) 

100 

RECORD! 

100 

Subprogram  Overhead  (non-generic,  cross  package) 


Number  of  iterations 


10000  *  10 


Time 

(microsec . ) 

Direction | 

Passed  | 

#  Passed |  Type  I  Size  of  | 

in  Call  |  Passed  | Passed  Varj 

1 

39.4 

1 

0 

1  1 

1 

1 

42.8 

I  | 

1 

| INTEGER  | 

1 

1 

45.8 

0  | 

1 

| INTEGER  | 

1 

1 

41.1 

I  0  | 

1 

| INTEGER  | 

1 

1 

43.4 

I  | 

10 

| INTEGER  | 

1 

1 

73.2 

0  | 

10 

(INTEGER  | 

1 

1 

108.7 

I  0  | 

10 

(INTEGER  | 

1 

1 

285.1 

I  | 

100 

| INTEGER  | 

1 

1 

472.0 

O  | 

100 

| INTEGER  | 

1 

1 

866.4 

I  0  1 

100 

| INTEGER  | 

1 

1 

42.2 

z  I 

1 

(ENUMERATION  | 

1 

1 

45.7 

o  | 

1 

(ENUMERATION  | 

1 

1 

41.1 

1  0  1 

1 

(ENUMERATION  | 

1 

1 

43.9 

I  1 

10 

(ENUMERATION  | 

1 

1 

72.0 

o  | 

10 

(ENUMERATION  | 

1 

1 

107.7 

I  0  1 

10 

(ENUMERATION  | 

1 

1 

271.4 

I  1 

100 

(ENUMERATION  I 

1 

1 

463.1 

o  | 

100 

(ENUMERATION  | 

1 

1 

847.9 

I  0  1 

100 

(ENUMERATION  | 

1 

1 

42.8 

I  1 

1 

| ARRAY  of  INTEGER  | 

1 

1 

1 

42.7 

o  | 

1 

| ARRAY  Of  INTEGER  | 

1 

1 

1 

39.1 

X  0  1 

1 

| ARRAY  of  INTEGER  | 

1 

1 

1 

44.1 

I  1 

1 

| ARRAY  of  INTEGER  | 

10 

1 

1 

42.4 

o  | 

1 

(ARRAY  Of  INTEGER  | 

10 

1 

1 

37.9 

I  O  1 

1 

1 ARRAY  Of  INTEGER  | 

10 

1 

1 

55.7 

I  1 

1 

| ARRAY  Of  INTEGER  I 

100 

1 

1 

56.7 

O  1 

1 

(ARRAY  Of  INTEGER  | 

100 

1 

1 

51.2 

I  o  | 

1 

| ARRAY  Of  INTEGER  | 

100 

1 

1 

43.6 

I  1 

1 

(RECORD  of  INTEGER  | 

1 

1 

1 

42.9 

O  1 

1 

(RECORD  of  INTEGER  | 

1 

1 

1 

38.8 

I  0  1 

1 

(RECORD  of  INTEGER  | 

1 

1 

1 

56.2 

I  f 

1 

(RECORD  of  INTEGER  | 

100 

1 

1 

55.6 

0  | 

1 

| RECORD  of  INTEGER  | 

100 

1 

1 

52.1 

I  0  I 

1 

| RECORD  of  INTEGER  ( 

100 

1 

. continued 
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54.3 

1 

I  | 

1 

| UNCONSTRAINED  ARRAY  | 

1 

58.9 

1 

0  | 

1 

| UNCONSTRAINED  ARRAY  | 

1 

49.8 

1 

I_0  | 

1 

| UNCONSTRAINED  ARRAY  | 

1 

67.5 

1 

I  1 

1 

i UNCONSTRAINED  ARRAY  | 

100 

71.8 

1 

o  | 

1 

| UNCONSTRAINED  ARRAY  | 

100 

62.5 

1 

I_0  1 

1 

(UNCONSTRAINED  ARRAY  | 

100 

42.6 

1 

I  1 

1 

| UNCONSTRAINED  RECORD | 

1 

43.9 

1 

o  | 

1 

(UNCONSTRAINED  RECORD | 

1 

38.8 

1 

I_0  I 

1 

(UNCONSTRAINED  RECORD | 

1 

55.3 

1 

I  1 

1 

(UNCONSTRAINED  RECORD | 

100 

56.1 

1 

o  t 

1 

(UNCONSTRAINED  RECORD | 

100 

52.1 

1 

I_0  1 

1 

(UNCONSTRAINED  RECORD | 

100 

CMU/SEI-87-TR-27 
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Subprogram  Overhead  (generic) 
Number  of  iterations  *  10000  *  10 


Time 

(microsec . ) 

Direction | 

Passed  | 

#  Passed  |  Type  1  Size  of  i 

in  Call  |  Passed  | Passed  Var| 

1 

-0.3 

1 

0 

1  1 

1 

1 

-5.3 

I  | 

1 

| INTEGER  | 

1 

1 

0.6 

0  { 

1 

| INTEGER  i 

1 

1 

0.5 

I_0  | 

1 

| INTEGER  | 

1 

1 

0.0 

I  | 

10 

(INTEGER  | 

1 

1 

0.1 

0  | 

10 

(INTEGER  | 

1 

1 

17.8 

I  0  | 

10 

| INTEGER  | 

1 

1 

112.9 

I  | 

100 

| INTEGER  | 

1 

1 

199.1 

o  i 

100 

| INTEGER  | 

1 

1 

304.4 

I  0  | 

100 

(INTEGER  | 

1 

1 

-4.9 

I  1 

1 

| ENUMERATION  | 

1 

1 

1.8 

o  | 

1 

| ENUMERATION  | 

1 

1 

-0.4 

I  o  1 

1 

(ENUMERATION  | 

1 

1 

-0.4 

I  1 

10 

(ENUMERATION  | 

1 

1 

-0.1 

o  | 

10 

(ENUMERATION  | 

1 

1 

10.1 

I  0  1 

10 

(ENUMERATION  | 

1 

1 

103.8 

I  1 

100 

(ENUMERATION  | 

1 

1 

191.7 

o  | 

100 

(ENUMERATION  I 

1 

1 

295.2 

I  0  1 

100 

(ENUMERATION  | 

1 

1 

-4.5 

I  1 

1 

| ARRAY  of  INTEGER  | 

1 

1 

1 

0.0 

o  | 

1 

| ARRAY  Of  INTEGER  | 

1 

1 

1 

0.1 

I  0  | 

1 

| ARRAY  Of  INTEGER  f 

1 

1 

1 

-2.9 

I  1 

1 

(ARRAY  Of  INTEGER  | 

10 

1 

1 

0.1 

O  | 

1 

| ARRAY  Of  INTEGER  | 

10 

1 

1 

0.8 

I  o  1 

1 

| ARRAY  Of  INTEGER  | 

10 

1 

1 

-4.1 

I  1 

1 

| ARRAY  Of  INTEGER  | 

100 

1 

1 

0.1 

o  | 

1 

| ARRAY  Of  INTEGER  | 

100 

1 

1 

0.0 

i_o  1 

1 

| ARRAY  Of  INTEGER  f 

100 

1 

1 

-4.4 

I  1 

1 

| RECORD  of  INTEGER  | 

1 

1 

1 

0.0 

0  | 

1 

| RECORD  Of  INTEGER  | 

1 

1 

1 

0.0 

I  0  | 

1 

| RECORD  of  INTEGER  | 

1 

1 

1 

-3.9 

I  1 

1 

| RECORD  of  INTEGER  j 

100 

1 

1 

0.0 

1 

| RECORD  Of  INTEGER  | 

100 

1 

1 

0.0 

■IB 

1 

| RECORD  of  INTEGER  | 

100 

1 
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Subprogram  Ovarhaad  (generic,  cross  package) 
Number  of  iterations  *  10000  *  10 


Tima 

(microsec. ) 

Direction | 

Passed  I 

1  Passed 

in  Call 

|  Type  I  Sise  of  | 

J  Passed  | Passed  Var | 

1 

14.3 

1 

0 

I  1 

1 

1 

15.1 

I  | 

1 

| INTEGER  | 

1 

1 

19.8 

0  | 

1 

| INTEGER  i 

1 

1 

24.6 

I  O  | 

1 

(INTEGER  | 

1 

1 

23.7 

I  | 

10 

| INTEGER  | 

1 

1 

51.6 

O  | 

10 

| INTEGER  | 

1 

1 

89.5 

I  0  | 

10 

(INTEGER  | 

1 

1 

277.4 

I  1 

100 

| INTEGER  ( 

1 

1 

442.2 

0  | 

100 

| INTEGER  | 

1 

1 

831.5 

I  0  1 

100 

| INTEGER  | 

1 

1 

14.4 

I  1 

1 

| enumeration  ( 

1 

1 

19.1 

0  | 

1 

(ENUMERATION  l 

1 

1 

24.7 

I  0  1 

1 

(ENUMERATION  ( 

1 

I 

25.8 

I  1 

10 

(ENUMERATION  | 

1 

1 

52.2 

o  | 

10 

(ENUMERATION  | 

1 

1 

89.3 

I  O  1 

10 

(ENUMERATION  | 

1 

1 

281.6 

I  1 

100 

(ENUMERATION  | 

1 

1 

422.5 

O  | 

100 

(ENUMERATION  | 

1 

1 

814.2 

I_0  1 

100 

(ENUMERATION  ( 

l 

1 

14.4 

I  1 

1 

|  ARRAY  Of  INTEGER  | 

1 

1 

1 

15.5 

O  | 

1 

| ARRAY  of  INTEGER  | 

1 

1 

1 

19.4 

X  O  1 

1 

|  ARRAY  Of  INTEGER  | 

1 

1 

1 

20.7 

I  1 

1 

| ARRAY  Of  INTEGER  ( 

10 

1 

1 

25.3 

O  1 

1 

| ARRAY  Of  INTEGER  | 

10 

1 

1 

22.4 

I  o  1 

1 

| ARRAY  Of  INTEGER  | 

10 

1 

l 

21.9 

I  1 

1 

(ARRAY  Of  INTEGER  | 

100 

1 

1 

25.0 

o  | 

1 

| ARRAY  Of  INTEGER  | 

100 

1 

1 

23.8 

I  0  1 

1 

| ARRAY  Of  INTEGER  | 

100 

1 

1 

16.1 

x  1 

1 

| RECORD  of  INTEGER  | 

1 

1 

1 

19.7 

0  | 

1 

| RECORD  of  INTEGER  | 

1 

1 

1 

19.6 

I  0  1 

1 

| RECORD  of  INTEGER  | 

1 

1 

1 

21.9 

x  1 

1 

| RECORD  of  INTEGER  | 

100 

1 

1 

24.1 

0  | 

1 

| RECORD  of  INTEGER  | 

100 

1 

1 

23.8 

I  O  1 

1 

| RECORD  of  INTEGER  | 

100 

1 
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A.i.  Memory  Management 

There  are  no  timing  results  produced  by  these  tests;  they  are  used  to  determine  whether  or 
not  garbage  collection  takes  place.  They  attempt  to  allocate  up  to  ten  million  integers  by 
successively  allocating  1000-integer  arrays  using  the  new  allocator.  Only  the  last  test  ex¬ 
plicitly  attempted  to  free  any  allocated  storage  (using  UNCHECKED_DEALLOCATION). 
The  tests  were  designed  either  to  report  how  much  storage  they  allocated  before  the  ex¬ 
pected  STORAGE_ERROR  exception  occurred,  or  a  message  saying  they  had  succeeded. 
Running  the  tests  confirmed  that  garbage  collection  did  not  occur;  reclamation  of  storage  is 
only  done  when  explicitly  requested.  This  may  be  the  reason  why  the  exception-handling 
tests  would  not  run  until  the  number  of  iterations  was  reduced  (see  the  Exception  Handling 
section). 

An  additional  test  included  with  the  memory  management  tests  uses  a  first  differencing 
scheme  to  determine  the  scheduling  discipline  of  the  target  operating  system.  This  test  was 
not  run  because  it  was  already  known  that  VAXELN  is  a  pre-emptive  priority-based  system. 
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Appendix  B:  Results:  PIWG  Benchmarks 

All  of  the  PIWG  tests,  with  the  exception  of  the  Hennessy  benchmark  (see  below),  ran  with¬ 
out  problems  and  without  the  need  to  tailor  the  VAXELN  system-build  process.  The  G  tests 
(Text JO  tests)  and  the  Z  tests  (compilation  tests)  were  not  run.  None  of  the  PIWG  tests 
produced  negative  numbers. 

The  output  of  each  PIWG  benchmark  program  contains  a  terse  description  of  the  feature 
being  measured.  For  any  further  details,  the  user  will  have  to  inspect  the  benchmark  code. 
The  reported  “Wail  Time”  is  based  on  calls  to  the  Calendar.Clock  function.  The  reported 
“CPU-Time"  is  based  on  calls  to  the  PIWG  function  CPU_TIME_CLOCK.  This  function  is 
intended  to  provide  an  interface  to  host-dependent  CPU-time  measurement  functions  on 
multi-user  systems  where  calls  to  Calendar.Clock  might  return  misleading  results.  For  the 
VAXELN  MicroVAX  tests,  the  basic  version  of  CPU_TIME_CLOCK,  which  simply  calls 
Calendar.Clock,  was  used. 

Because  of  the  issue  of  the  accuracy  of  PIWG  results  (see  Problems  Encountered  and  Les¬ 
sons  Learned  section),  the  table  below  is  provided.  Note  that  the  actual  iterations  of  the 
benchmarks  are  100  times  greater  than  the  reported  iteration  counts.  The  reported  counts 
are  only  for  the  main  loop  enclosing  the  control  and  test  loops;  these  latter  loops  alway  iter¬ 
ate  100  times.  The  accuracy  delta  is  computed  by  dividing  the  resolution  of  the 
Calendar.Clock  function  (10  milliseconds)  by  the  actual  number  of  iterations. 


Reported 

Actual 

Accuracy 

Iteration 

iterations 

Delta 

Count 

in  Microseconds 

1 

100 

100.0 

2 

200 

50.0 

4 

400 

25.0 

8 

800 

12.5 

16 

1600 

6.25 

32 

3200 

3.125 

64 

6400 

1.5625 

128 

12800 

0.781250 

256 

25600 

0.390625 

B.a.  Composite  Benchmarks 

B.0.0.1.  The  Dhrystone  Benchmark 

This  is  a  version  of  the  benchmark  described  in  [1 1]. 

1.1710  is  tima  in  milliseconds  for  one  Dhrystone 

B.0.0.2.  The  Whetstone  Benchmark 

Two  versions  of  the  Whetstone  benchmark  [5]  are  provided.  One  uses  the  math  library  sup¬ 
plied  by  the  vendor  (with  FLOAT_MATH_LIB  for  the  VAXELN  Ada  compiler);  the  other  has 
the  math  functions  coded  within  the  benchmark  program  so  that  the  test  can  be  run  even 
when  a  math  library  is  not  supplied.  "KWIPS"  means  Kilo  Whetstones  Per  Second. 

ADA  Whetstone  benchmark 

A000092  using  manufacturer' s  math  routines 

Average  time  per  cycle  :  808.32  milliseconds 

Average  Whetstone  rating  :  1237  KWIPS 


ADA  Whetstone  benchmark 

A000093  using  standard  internal  math  routines 

Average  time  per  cycle  :  1046.63  milliseconds 

Average  Whetstone  rating  :  955  XWZPS 

B  .0.0.3.  The  Hennessy  Benchmark 

This  is  a  collection  of  benchmarks  that  are  relatively  short  in  terms  of  program  size  and 
execution  time.  Named  after  the  person  who  gathered  the  tests,  it  includes  such  well-known 
programming  problems  as  the  Eight  Queens  problem,  the  Tower  of  Hanoi,  Quicksort,  Bub¬ 
ble  Sort,  Fast  Fourier  Transform,  and  Ackermann’s  Function.  The  Hennessy  benchmark, 
known  as  PIWG  A000094,  was  the  only  PIWG  benchmark  that  failed  to  execute;  it  crashed 
with  a  STORAGE_ERROR  exception.  Initial  attempts  to  resolve  the  problem  were  unsuc¬ 
cessful.  It  is  believed,  however,  that  the  solution  lies  in  simply  finding  the  right  settings  for 
the  storage  parameters  of  the  VAXELN  build  process. 
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B.b.  Task  Creation 


Test  nans:  C000001  Class  name:  Tasking 

CPU  tins:  9400.0  aiicrosaconds 

Wall  tins:  9400.0  microseconds  Iteration  count:  2 

Test  description: 

Task  create  and  teminate  measurement 

with  one  task,  no  entries,  when  task  is  in  a  procedure 

using  a  task  type  in  a  package,  no  select  statement,  no  loop 


Test  name:  C000002  Class  name:  Tasking 

CPU  time:  9549.9  microseconds 

Wall  time:  9549.9  microseconds  Iteration  count:  2 

Test  description: 

Task  create  and  terminate  time  measurement 

with  one  task,  no  entries,  when  task  is  in  a  procedure 

task  defined  and  used  in  procedure,  no  select  statement,  no  loop 


Test  name:  C000003  Class  name:  Tasking 

CPU  time:  9599.9  microseconds 

Wall  time:  9599.9  microseconds  Iteration  count:  2 

Test  description: 

Task  create  and  terminate  time  measurement 
task  is  in  declare  block  of  main  procedure 
one  task,  no  entries,  task  is  in  the  loop 
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B.c.  Dynamic  Storage  Allocation 

Test  naaw:  D000001  Class  name:  Allocation 

CPU  tias:  38.3  aicrossconds 

Wall  tins:  38.3  aicrossconds  Iteration  count:  128 

Test  description: 

Dynamic  array  allocation,  use  and  deallocation  time  measurement 
dynastic  array  elaboration,  1000  integers  in  a  procedure 
get  space  and  free  it  in  the  procedure  on  each  call 


Test  name:  D000002  Class  name:  Allocation 

CPU  time:  4225.0  sticroseconds 

Wall  time:  4225.0  microseconds  Iteration  count:  4 

Test  description: 

Dynastic  array  elaboration  and  initialisation  time  sieasurement 
allocation,  initialisation,  use  and  deallocation 
1000  integers  initialized  by  others*>l 


Test  name:  D000003  Class  name:  Allocation 

CPU  time:  23.4  microseconds 

Wall  time:  23.4  microseconds  Iteration  count:  128 

Test  description: 

Dynamic  record  allocation  and  deallocation  time  measurement 

elaborating,  allocating  and  deallocating 

record  containing  a  dynamic  array  of  1000  integers 


Test  name:  D000004  Class  name:  Allocation 

CPU  time:  5350.3  microseconds 

Wall  time :  5350 . 3  microseconds  Iteration  count :  2 

Test  description: 

Dynamic  record  allocation  and  deallocation  time  measurement 
elaborating,  initializing  by  (DYNAMIC_SIZE, (others=>l)) 
record  containing  a  dynamic  array  of  1000  Integers 
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B.d.  Exception  Handling 

There  is  no  E000003  test  in  the  PIWG  8/31/86  suite. 

Teat  name:  X000001  Class  name:  Exception 

CPU  time:  825.0  microseconds 

Wall  time:  825.0  microseconds  Iteration  count:  16 

Test  description: 

Time  to  raise  and  handle  an  exception 
exception  defined  locally  and  handled  locally 


Test  name:  X000002  Class  name:  Exception 

CPU  time:  1093.8  microseconds 

Wall  time:  1093.8  microseconds  Iteration  count:  16 

Test  description: 

Exception  raise  and  handle  timing  measurement 
when  exception  is  in  a  procedure  in  a  package 


Test  name:  E000004  Class  name:  Procedure 

CPU  time:  881.2  microseconds 

Wall  time:  881.2  microseconds  Iteration  count:  16 

Test  description: 

Exception  raise  and  handle  timing  measurement 
when  exception  is  in  a  package  four  deep 
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Class  name:  Style 
Iteration  count:  256 


B.e.  Coding  Style 

Test  name:  7000001 

CPU  time:  3.9  microseconds 

Wall  time:  4.3  microseconds 

Test  description: 

Time  to  set  a  boolean  flag  using  a  logical  equation 
a  local  and  a  global  integer  are  compared 
compare  this  test  with  7000002 


Test  name:  F000002  Class  name:  Style 

CPU  time:  2.7  microseconds 

Wall  time:  2.7  microseconds  Iteration  count:  256 

Test  description: 

Time  to  set  a  boolean  flag  using  an  "if"  test 
a  local  and  a  global  integer  are  compared 
compare  this  test  with  F000001 


B.f.  Loop  Overhead 

Test  name:  L000001  Class  name:  Iteration 

CPU  time:  2.0  microseconds 

Wall  time:  2.0  microseconds  Iteration  count:  2 

Test  description: 

Simple  "for"  loop  time 
for  I  in  1  . .  100  loop 
time  reported  is  for  once  through  loop 


Test  name:  L000002  Class  name:  Iteration 

CPU  time:  2.5  microseconds 

Wall  time :  2.5  microseconds  Iteration  count :  2 

Test  description: 

Sinqple  "while"  loop  time 
while  I  <*  100  loop 

time  reported  is  for  once  through  loop 


Test  name:  L000003  Class  name:  Iteration 

CPU  time :  2.0  microseconds 

Wall  time:  2.0  microseconds  Iteration  count:  2 

Test  description: 

Simple  "exit"  loop  time 
loop  I: *1+1;  exit  when  l>100;  end  loop; 
time  reported  is  for  once  through  loop 


B.g.  Procedure  Calls 

There  is  no  P000008  or  P000009  test  in  the  PIWG  8/31/86  suite. 

Test  nan*:  P000001  Class  name:  Procedure 

CPU  time:  0.4  microseconds 

Wall  time :  0.4  siieroseconds  Iteration  count:  256 

Test  description: 

Procedure  call  and  return  time  (may  be  zero  if  automatic  inlining) 
procedure  is  local 
no  parameters 


microseconds 

microseconds 


Test  name:  P000002 

CPU  time:  54.7 

Wall  time:  55.5 

Test  description: 

Procedure  call  and  return  time 
procedure  is  local,  no  parameters 
when  procedure  is  not  inlinable 


Class  name:  Procedure 
Iteration  count:  128 


Class  name: 


microseconds 

microseconds 


Test  name:  P000003 

CPU  time:  42.2 

Wall  time:  42.2 

Test  description: 

Procedure  call  and  return  time  measurement 

the  procedure  is  in  a  separately  compiled  package 

compare  to  P000002 


Procedure 


Iteration  count: 


128 


microseconds 

microseconds 


Test  name:  P000004 

CPU  time :  0.0 

Wall  time:  0.0 

Test  description: 

Procedure  call  and  return  time  measurement 
procedure  is  in  a  separately  conpiled  package 
pragma  INLINE  used 
compare  to  P000001 


Class  name:  Procedure 


Iteration  count : 


256 


microseconds 

microseconds 


Test  name:  P000005 

CPU  time:  44.5 

Wall  time:  44.5 

Test  description: 

Procedure  call  and  return  time  measurement 
procedure  is  in  a  separately  cospiled  package 
one  parameter,  in  INTEGER 


Class  name:  Procedure 


Iteration  count : 


128 
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Test  name :  P000006  Class  name:  Procedure 

CPU  time:  48.4  microseconds 

Wall  time:  48.4  microseconds  Iteration  count:  128 

Test  description: 

Procedure  call  and  return  time  measurement 
procedure  is  in  a  separately  compiled  package 
one  parameter,  out  INTEGER 

Test  name:  P000007  Class  name:  Procedure 

CPU  time :  51.6  microseconds 

Wall  time:  51.6  microseconds  Iteration  count:  128 

Test  description: 

Procedure  call  and  return  time  measurement 
procedure  is  in  a  separately  compiled  package 
one  parameter,  in  out  INTEGER 

Test  name:  P000010  Class  name:  Procedure 

CPU  time:  74.2  microseconds 

Wall  time:  74.2  imicroseconds  Iteration  count:  128 

Test  description: 

Procedure  call  and  return  time  measurement 
Compare  to  P000005 
10  parameters,  in  INTEGER 

Test  name:  P000011  Class  name:  Procedure 

CPU  time:  106.2  microseconds 

Wall  time:  106.2  microseconds  Iteration  count:  64 

Test  description: 

Procedure  call  and  return  time  measurement 
compare  to  P000005,  P000010 
20  parameters,  in  INTEGER 

Test  name:  P000012  Class  name:  Procedure 

CPU  time :  65.6  microseconds 

Wall  time:  65.6  microseconds  Iteration  count:  128 

Test  description: 

Procedure  call  and  return  time  measurement 

Compare  with  P000010  (discrete  vs  composite  parameters  ) 

10  parameters,  in  MY_REC0RD  a  three  component  record 

Test  name:  P000013  Class  name:  Procedure 

CPU  time:  93.8  microseconds 

Wall  time:  93.8  microseconds  Iteration  count:  64 

Test  description: 

Procedure  call  and  return  time  measurement 

twenty  composite  "in"  parameters 

package  body  is  compiled  after  the  spec  is  used 
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B.h.  Task  Rendezvous 


Test  n ame:  T000001  Class  name:  Tasking 

CPU  tins:  1662.5  microseconds 

Wall  tins:  1662.5  micsoseconds  Iteration  count:  8 

Test  description: 

Minimum  rendezvous,  entry  call  and  return  time 
one  task,  one  entry,  task  inside  procedure 
no  select 


Test  name:  T000002  Class  name:  Tasking 

CPU  time:  1637.5  microseconds 

Wall  time:  1650.0  microseconds  Iteration  count:  8 

Test  description: 

Task  entry  call  and  return  time  measured 

one  task  active,  one  entry  in  task,  task  in  a  package 

no  select  statement 


Test  name:  T000003  Class  name:  Tasking 

CPU  time:  1675.0  microseconds 

Wall  time:  1675.0  microseconds  Iteration  count:  4 

Test  description: 

Task  entry  call  and  return  time  measured 

two  tasks  active,  one  entry  per  task,  tasks  in  a  package 

no  select  statement 


Test  name:  T000004  Class  name:  Tasking 

CPU  time:  1837.5  microseconds 

Wall  time:  1837.5  microseconds  Iteration  count:  4 

Test  description: 

Task  entry  call  and  return  time  measured 

one  task  active,  two  entries,  tasks  in  a  package 

using  select  statement 


Test  name:  T000005  Class  name:  Tasking 

CPU  time:  1689.9  microseconds 

Wall  time:  1689.9  microseconds  Iteration  count:  1 

Test  description: 

Task  entry  call  and  return  time  measured 

ten  tasks  active,  one  entry  per  task,  tasks  in  a  package 

no  select  statement 
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Test  nans:  T000006  Class  name:  Tasking 

CPU  time:  2429.9  microseconds 

Wall  time:  2419.9  microseconds  Iteration  count:  1 

Test  description: 

Task  entry  call  and  return  time  measurement 
one  task  with  ten  entries,  task  in  a  package 
one  select  statement,  cooqpare  to  T000005 


Test  name:  T000007 

CPU  time:  1612. 5 

Wall  time:  1600.0 

Test  description: 
Minimum  rendezvous, 
one  task  one  entry 
no  select 


Class  name:  Tasking 

microseconds 

microseconds  Iteration  count :  8 

entry  call  and  return  time 
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